Evolving Deep DenseBlock Architecture Ensembles for Image Classification

Automatic deep architecture generation is a challenging task, owing to the large number of controlling parameters inherent in the construction of deep networks. The combination of these parameters leads to the creation of large, complex search spaces that are feasibly impossible to properly navigate without a huge amount of resources for parallelisation. To deal with such challenges, in this research we propose a Swarm Optimised DenseBlock Architecture Ensemble (SODBAE) method, a joint optimisation and training process that explores a constrained search space over a skeleton DenseBlock Convolutional Neural Network (CNN) architecture. Specifically, we employ novel weight inheritance learning mechanisms, a DenseBlock skeleton architecture, as well as adaptive Particle Swarm Optimisation (PSO) with cosine search coefficients to devise networks whilst maintaining practical computational costs. Moreover, the architecture design takes advantage of recent advancements of the concepts of residual connections and dense connectivity, in order to yield CNN models with a much wider variety of structural variations. The proposed weight inheritance learning schemes perform joint optimisation and training of the architectures to reduce the computational costs. Being evaluated using the CIFAR-10 dataset, the proposed model shows great superiority in classification performance over other state-of-the-art methods while illustrating a greater versatility in architecture generation

Evolving and Ensembling Deep CNN Architectures for Image Classification

Deep Convolutional Neural Networks (CNNs) have traditionally been hand-designed owing to the complexity of their construction and the computational requirements of their training. Recently however, there has been an increase in research interest towards automatically designing deep CNNs for specific tasks. Ensembling has been shown to effectively increase the performance of deep CNNs, although usually with a duplication of work and therefore a large increase in computational resources required. In this paper we present a method for automatically designing and ensembling deep CNN models with a central weight repository to avoid work duplication. The models are trained and optimised together using particle swarm optimisation (PSO), with architecture convergence encouraged. At the conclusion of the joint optimisation and training process a base model nomination method is used to determine the best candidates for the ensemble. Two base model nomination methods are proposed, one using the local best particle positions from the PSO process, and one using the contents of the central weight repository. Once the base model pool has been created, the individual models inherit their parameters from the central weight repository and are then finetuned and ensembled in order to create a final system. We evaluate our system on the CIFAR-10 classification dataset and demonstrate improved results over the single global best model suggested by the optimisation process, with a minor increase in resources required by the finetuning process. Our system achieves an error rate of 4.27% on the CIFAR-10 image classification task with only 36 hours of combined optimisation and training on a single NVIDIA GTX 1080Ti GPU

Evolving Ensemble Models for Image Segmentation Using Enhanced Particle Swarm Optimization

In this paper, we propose particle swarm optimization (PSO)-enhanced ensemble deep neural networks and hybrid clustering models for skin lesion segmentation. A PSO variant is proposed, which embeds diverse search actions including simulated annealing, levy flight, helix behavior, modified PSO, and differential evolution operations with spiral search coefficients. These search actions work in a cascade manner to not only equip each individual with different search operations throughout the search process but also assign distinctive search actions to different particles simultaneously in every single iteration. The proposed PSO variant is used to optimize the learning hyper-parameters of convolutional neural networks (CNNs) and the cluster centroids of classical Fuzzy C-Means clustering respectively to overcome performance barriers. Ensemble deep networks and hybrid clustering models are subsequently constructed based on the optimized CNN and hybrid clustering segmenters for lesion segmentation. We evaluate the proposed ensemble models using three skin lesion databases, i.e., PH2, ISIC 2017, and Dermofit Image Library, and a blood cancer data set, i.e., ALL-IDB2. The empirical results indicate that our models outperform other hybrid ensemble clustering models combined with advanced PSO variants, as well as state-of-the-art deep networks in the literature for diverse challenging image segmentation tasks

OzBot TM - Haptic augmentation of a teleoperated robotic platform for search and rescue operations

A continued increase in computing power, sensor capability, software functionality, immersive interfaces and hardware modularity has given robot designers seemingly endless potential in the area of mobile robotics.  While some mobile robotic system designers are focusing on expensive, full-featured platforms, developers are realising the advantages of emerging technology in providing small, low-cost mobile reconnaissance vehicles as expendable teleoperated robotic systems.  The OzBotTM mobile reconnaissance platform presents one such system.  The design objectives of the OzBotTM platform focus on the development of inexpensive, lightweight carry-case sized robots for search and rescue operations, law enforcement scenarios and hazardous environment inspection.  The incorporation of Haptic augmentation provides the teleoperator with improved task immersion for an outdoor search and rescue scenario.  Achieved in cooperation with law enforcement agencies within Australia, this paper discusses the performance of the first four revisions of the OzBotTM platform

Counseling versus antidepressant therapy for the treatment of mild to moderate depression in primary care: economic analysis.

OBJECTIVE: To compare the cost-effectiveness of generic psychological therapy (counseling) with routinely prescribed antidepressant drugs in a naturalistic general practice setting for a follow-up period of 12 months. METHODS: Economic analysis alongside a randomized clinical trial with patient preference arm. Comparison of depression-related health service costs at 12 months. Cost-effectiveness analysis of bootstrapped trial data using net monetary benefits and acceptability curves. RESULTS: No significant difference between the mean observed costs of patients randomized to antidepressants or to counseling (342 pounds sterling vs 302 pounds sterling , p = .56 [t test]). If decision makers are not willing to pay more for additional benefits (value placed on extra patient with good outcome, denoted by K, is zero), then we find little difference between the treatment modalities in terms of cost-effectiveness. If decision makers do place value on additional benefit (K > 0 pounds sterling), then the antidepressant group becomes more likely to be cost-effective. This likelihood is in excess of 90% where decision makers are prepared to pay an additional 2,000 pounds sterling or more per additional patient with a good global outcome. The mean values for incremental net monetary benefits (INMB) from antidepressants are substantial for higher values of K (INMB = 406 pounds sterling when K = 2,500 pounds sterling). CONCLUSION: For a small proportion of patients, the counseling intervention (as specified in this trial) is a dominant cost-effective strategy. For a larger proportion of patients, the antidepressant intervention (as specified in this trial) is the dominant cost-effective strategy. For the remaining group of patients, cost-effectiveness depends on the value of K. Since we cannot observe K, acceptability curves are a useful way to inform decision makers

Revisiting cannabinoid receptor 2 expression and function in murine retina.

The cannabinoid receptor CB2 plays a significant role in the regulation of immune function whereas neuronal expression remains a subject of contention. Multiple studies have described CB2 in retina and a recent study showed that CB2 deletion altered retinal visual processing. We revisited CB2 expression using immunohistochemistry and a recently developed CB2-eGFP reporter mouse. We examined the consequence of acute vs. prolonged CB2 deactivation on the electroretinogram (ERG) responses. We also examined lipidomics in CB2 knockout mice and potential changes in microglia using Scholl analysis. Consistent with a published report, in CB2 receptor knockout mice see an increased ERG scotopic a-wave, as well as stronger responses in dark adapted cone-driven ON bipolar cells and, to a lesser extent cone-driven ON bipolar cells early in light adaptation. Significantly, however, acute block with CB2 antagonist, AM630, did not mimic the results observed in the CB2 knockout mice whereas chronic (7 days) block did. Immunohistochemical studies show no CB2 in retina under non-pathological conditions, even with published antibodies. Retinal CB2–eGFP reporter signal is minimal under baseline conditions but upregulated by intraocular injection of either LPS or carrageenan. CB2 knockout mice see modest declines in a broad spectrum of cannabinoid-related lipids. The numbers and morphology of microglia were unaltered. In summary minimal CB2 expression is seen in healthy retina. CB2 appears to be upregulated under pathological conditions. Previously reported functional consequences of CB2 deletion are an adaptive response to prolonged blockade of these receptors. CB2 therefore impacts retinal signaling but perhaps in an indirect, potentially extra-ocular fashion.post-print8285 K

Atlanto-axial rotatory fixation in a girl with Spondylocarpotarsal synostosis syndrome

We report a 15-year-old girl who presented with spinal malsegmentation, associated with other skeletal anomalies. The spinal malsegmentation was subsequently discovered to be part of the spondylocarpotarsal synostosis syndrome. In addition, a distinctive craniocervical malformation was identified, which included atlanto-axial rotatory fixation. The clinical and the radiographic findings are described, and we emphasise the importance of computerised tomography to characterize the craniocervical malformation complex. To the best of our knowledge, this is the first clinical report of a child with spondylocarpotarsal synostosis associated with atlanto-axial rotatory fixation

Slow relaxation due to optimization and restructuring: Solution on a hierarchical lattice

Motivated by the large strain shear of loose granular materials we introduced a model which consists of consecutive optimization and restructuring steps leading to a self organization of a density field. The extensive connections to other models of statistical phyics are discussed. We investigate our model on a hierarchical lattice which allows an exact asymptotic renormalization treatment. A surprisingly close analogy is observed between the simulation results on the regular and the hierarchical lattices. The dynamics is characterized by the breakdown of ergodicity, by unusual system size effects in the development of the average density as well as by the age distribution, the latter showing multifractal properties.Comment: 11 pages, 7 figures revtex, submitted to PRE see also: cond-mat/020920

Symposium abstracts: Thresholds and permeability in performance

Publisher PD

Fair winds and following seas remotely: modifying perceptions of fieldwork as a requirement in marine science to aid in diversifying the discipline

Pursuing an academic career in marine science requires a range of skills that can be applied across different contexts, including experimental or computational proficiency, policy engagement, teaching, and seagoing fieldwork. The tendency to advertise careers in marine science with imagery of research expeditions results in the perception that it is a requirement for a career in marine science, an indicator of competitiveness in this discipline. Historically, those participating in remote fieldwork over extended periods of time were perceived as “adventurous explorers, with a strong bias towards western, able-bodied men” (Nash et al., 2019). Use of imagery reinforcing such notions for marine scientists fails to recognize that this perception can be discouraging to individuals from other backgrounds who may be excluded from the discipline by a range of real and perceived participatory barriers. Such exclusionary factors include: caring responsibilities, physical mobility, challenging social environments, isolating and physically uncomfortable working environments, mental health challenges, and access to opportunity (Giles et al., 2020). Such barriers disproportionately affect diverse, underrepresented, and marginalized groups, who may therefore struggle to identify with marine science as a potential discipline in which to pursue a successful career. Current work toward achieving net zero targets within ocean research emphasizes the use of autonomous vehicles as alternatives to ocean-going ships (Storey, 2023), and the proposed concept of digital twinning would incorporate similar remote technology coupled with simulations and shore-based decision-making. The concept of digital twinning refers to the use of responsive autonomous platforms that can both collect data and be operated in response to that data, which could provide a non-field-based approach to delivering marine science while also potentially expanding the opportunities available for individuals not able or interested in working in the field. In distinguishing digital twinning from current approaches such as data assimilating models, Kritzinger et al. (2018) note the importance of a two-way data flow between the physical environment and its virtual representation, called a “digital twin,” which, for example, may lead to changes in deployment strategy or data collection by researchers. Because these twins can be controlled and simulated anywhere with access to sufficient computing power, shore-based individuals can interact with a virtual version of the physical environment without being physically present at sea. The technology to support a fully realized digital twin of the ocean is still under development, but its use would require a broader range of skills and roles in the discipline, many of which are not accurately conveyed by the prevailing marketing of field-based disciplines (see Mol and Atchinson, 2019, regarding geosciences). In order to fully integrate this new approach into marine science, employment of individuals with experience and training across a wide range of disciplines from software engineering to traditional field sampling is essential while also presenting the potential for making marine science more inclusive. Individuals for whom working at sea is not possible and/or desirable would be able to make equally valid contributions to such research projects via digital routes, without facing the many barriers fieldwork may present. This study explores the expectations of marine scientists, from both early and more established career stages, around the importance of field experience as a precursor or requirement for a successful marine science career, and also examines the advantages and disadvantages of using digital twinning as a complement to traditional field-based marine science